Tootfinder

Numerical Fuzz: A Type System for Rounding Error Analysis
Ariel E. Kellison, Justin Hsu
https://arxiv.org/abs/2405.04612 https://arxi…

Numerical Fuzz: A Type System for Rounding Error Analysis
Algorithms operating on real numbers are implemented as floating-point computations in practice, but floating-point operations introduce roundoff errors that can degrade the accuracy of the result. We propose $Λ_{num}$, a functional programming language with a type system that can express quantitative bounds on roundoff error. Our type system combines a sensitivity analysis, enforced through a linear typing discipline, with a novel graded monad to track the accumulation of roundoff errors. We …

> Przez 10 lat nie zgłoszono żadnego błędu, pomimo intensywnego użycia w Pythonie i PHP.
(z https://www.bytereef.org/mpdecimal/, tłum. własne)
Ja znajduję 2 błędy przy przygotowywaniu paczki.
No dobra, pewnie nie mieli na myśli błędów w systemie budowania.

Google's Education VP wants us to believe AI is the classroom's new calculator, but this is a terrible analogy:
1. We know how calculators produce their results.
2. You can check a calculator's answer using pretty much the same algorithm it uses.
3. Rare floating point errors aside, calculators do not invent false answers.
4. Calculators are based on math principles; LLMs are based on no principles.

This https://arxiv.org/abs/2404.04081 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_qu…

Clock offset recovery with sublinear complexity enables synchronization on low-level hardware for quantum key distribution
We introduce iQSync, a clock offset recovery method designed for implementation on low-level hardware, such as FPGAs or microcontrollers, for quantum key distribution (QKD). iQSync requires minimal memory, only a simple instruction set (e.g. no floating-point operations), and can be evaluated with sublinear time complexity, typically involving no more than a few thousand iterations of a simple loop. Furthermore, iQSync allows for a precise clock offset recovery within few seconds, even for larg…

Effective Quadratic Error Bounds for Floating-Point Algorithms Computing the Hypotenuse Function
Jean-Michel Muller, Bruno Salvy
https://arxiv.org/abs/2405.03588

Effective Quadratic Error Bounds for Floating-Point Algorithms Computing the Hypotenuse Function
We provide tools to help automate the error analysis of algorithms that evaluate simple functions over the floating-point numbers. The aim is to obtain tight relative error bounds for these algorithms, expressed as a function of the unit round-off. Due to the discrete nature of the set of floating-point numbers, the largest errors are often intrinsically "arithmetic" in the sense that their appearance may depend on specific bit patterns in the binary representations of intermediate variables, w…

Performance of H-Matrix-Vector Multiplication with Floating Point Compression
Ronald Kriemann
https://arxiv.org/abs/2405.03456 https://

Performance of H-Matrix-Vector Multiplication with Floating Point Compression
Matrix-vector multiplication forms the basis of many iterative solution algorithms and as such is an important algorithm also for hierarchical matrices. However, due to its low computational intensity, its performance is typically limited by the available memory bandwidth. By optimizing the storage representation of the data within such matrices, this limitation can be lifted and the performance increased. This applies not only to hierarchical matrices but for also for other low-rank approximat…

Privacy-Aware Semantic Cache for Large Language Models
Waris GillVirginia Tech, USA, Mohamed ElidrisiCisco, USA, Pallavi KalapatapuCisco, USA, Ali AnwarUniversity of Minnesota, Minneapolis, USA, Muhammad Ali GulzarVirginia Tech, USA
https://arxiv.org/abs/2403.02694

Privacy-Aware Semantic Cache for Large Language Models
Large Language Models (LLMs) like ChatGPT, Google Bard, Claude, and Llama 2 have revolutionized natural language processing and search engine dynamics. However, these models incur exceptionally high computational costs. For instance, GPT-3 consists of 175 billion parameters and inference on these models also demands billions of floating-point operations. Caching is a natural solution to reduce LLM inference costs on repeated queries. However, existing caching methods are incapable of finding se…

AFPR-CIM: An Analog-Domain Floating-Point RRAM-based Compute-In-Memory Architecture with Dynamic Range Adaptive FP-ADC
Haobo Liu, Zhengyang Qian, Wei Wu, Hongwei Ren, Zhiwei Liu, Leibin Ni
https://arxiv.org/abs/2402.13798

AFPR-CIM: An Analog-Domain Floating-Point RRAM-based Compute-In-Memory Architecture with Dynamic Range Adaptive FP-ADC
Power consumption has become the major concern in neural network accelerators for edge devices. The novel non-volatile-memory (NVM) based computing-in-memory (CIM) architecture has shown great potential for better energy efficiency. However, most of the recent NVM-CIM solutions mainly focus on fixed-point calculation and are not applicable to floating-point (FP) processing. In this paper, we propose an analog-domain floating-point CIM architecture (AFPR-CIM) based on resistive random-access mem…

Just realised #bash can't do floating point arithmetic, so I have to use bc.
I actually loved bash's "(( EXPRESSION ))" operator and mistakenly thought it could do floating point as well.

Zero-Knowledge Location Privacy via Accurate Floating Point SNARKs
Jens Ernstberger, Chengru Zhang, Luca Ciprian, Philipp Jovanovic, Sebastian Steinhorst
https://arxiv.org/abs/2404.14983

Zero-Knowledge Location Privacy via Accurate Floating Point SNARKs
This paper introduces Zero-Knowledge Location Privacy (ZKLP), enabling users to prove to third parties that they are within a specified geographical region while not disclosing their exact location. ZKLP supports varying levels of granularity, allowing for customization depending on the use case. To realize ZKLP, we introduce the first set of Zero-Knowledge Proof (ZKP) circuits that are fully compliant to the IEEE 754 standard for floating-point arithmetic. Our results demonstrate that our fl…

FlowFPX: Nimble Tools for Debugging Floating-Point Exceptions
Taylor Allred, Xinyi Li, Ashton Wiersdorf, Ben Greenman, Ganesh Gopalakrishnan
https://arxiv.org/abs/2403.15632

FlowFPX: Nimble Tools for Debugging Floating-Point Exceptions
Reliable numerical computations are central to scientific computing, but the floating-point arithmetic that enables large-scale models is error-prone. Numeric exceptions are a common occurrence and can propagate through code, leading to flawed results. This paper presents FlowFPX, a toolkit for systematically debugging floating-point exceptions by recording their flow, coalescing exception contexts, and fuzzing in select locations. These tools help scientists discover when exceptions happen and…

When I ordered our wall-print, I also ordered an A4 sized print of the photo on High Quality paper (Hahnemühle Baryta).
And I must admit: I'm really amazed! Nice, thick paper, some structure. semi glossy. It really feels like quality. Oh and it's my wife's favourite photo. I'll try to post one or two more later. Maybe I can show more of it.
The photo was taken in #Ireland

This https://arxiv.org/abs/2404.02826 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIT_…

An Error-Bounded Lossy Compression Method with Bit-Adaptive Quantization for Particle Data
This paper presents error-bounded lossy compression tailored for particle datasets from diverse scientific applications in cosmology, fluid dynamics, and fusion energy sciences. As today's high-performance computing capabilities advance, these datasets often reach trillions of points, posing significant visualization, analysis, and storage challenges. While error-bounded lossy compression makes it possible to represent floating-point values with strict pointwise accuracy guarantees, the lack of…

What Every Computer Scientist Should Know About Floating-Point Arithmetic #floats

A didactic example: floating point maths is dodgy
julia> sqrt(2)
1.4142135623730951
julia> 2/sqrt(2)
1.414213562373095
julia> 2/sqrt(2) == sqrt(2)
false
>>> math.sqrt(2)
1.4142135623730951
>>> 2/math.sqrt(2)
1.414213562373095
>>> 2/math.sqrt(2) == math.sqrt(2)
False
. di .16f sqrt(2)
1.4142135623730951
. di .16f 2/sqrt(2)
1.4142135623730949
. assert 2/sqrt(2) == sq…

This https://arxiv.org/abs/2402.02301 has been replaced.
link: https://scholar.google.com/scholar?q=a

MATLAB Simulator of Level-Index Arithmetic
Level-index arithmetic appeared in the 1980s. One of its principal purposes is to abolish the issues caused by underflows and overflows in floating point. However, level-index arithmetic does not expand the set of numbers but spaces out the numbers of large magnitude even more than floating-point representations to move the infinities further away from zero: gaps between numbers on both ends of the range become very large. We revisit level index by presenting a custom precision simulator in MAT…

Interesting data, nice visualization here: https://ourworldindata.org/grapher/artificial-intelligence-training-computation

Computation used to train notable artificial intelligence systems
Computation is measured in total petaFLOP, which is 10¹⁵ floating-point operations estimated from AI literature, albeit with some uncertainty. Estimates are expected to be accurate within a factor of 2, or a factor of 5 for recent undisclosed models like GPT-4.

FlowFPX: Nimble Tools for Debugging Floating-Point Exceptions
Taylor Allred, Xinyi Li, Ashton Wiersdorf, Ben Greenman, Ganesh Gopalakrishnan
https://arxiv.org/abs/2403.15632

FlowFPX: Nimble Tools for Debugging Floating-Point Exceptions
Reliable numerical computations are central to scientific computing, but the floating-point arithmetic that enables large-scale models is error-prone. Numeric exceptions are a common occurrence and can propagate through code, leading to flawed results. This paper presents FlowFPX, a toolkit for systematically debugging floating-point exceptions by recording their flow, coalescing exception contexts, and fuzzing in select locations. These tools help scientists discover when exceptions happen and…

https://slow-bros.com/harold-halibut
Harold Halibut (Multi, XP'd on PC) Experience life on FEDORA, a spaceship crashed and floating in an endless ocean, from the POV of Harold, Fedora's Everyman janitor.
Do you like Point & Click/Adventure games? Play this. It's that…

This https://arxiv.org/abs/2403.05465 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csAR_…

Algorithm-Hardware Co-Design of Distribution-Aware Logarithmic-Posit Encodings for Efficient DNN Inference
Traditional Deep Neural Network (DNN) quantization methods using integer, fixed-point, or floating-point data types struggle to capture diverse DNN parameter distributions at low precision, and often require large silicon overhead and intensive quantization-aware training. In this study, we introduce Logarithmic Posits (LP), an adaptive, hardware-friendly data type inspired by posits that dynamically adapts to DNN weight/activation distributions by parameterizing LP bit fields. We also develop …

An Error-Bounded Lossy Compression Method with Bit-Adaptive Quantization for Particle Data
Congrong Ren, Sheng Di, Longtao Zhang, Kai Zhao, Hanqi Guo
https://arxiv.org/abs/2404.02826

An Error-Bounded Lossy Compression Method with Bit-Adaptive Quantization for Particle Data
This paper presents error-bounded lossy compression tailored for particle datasets from diverse scientific applications in cosmology, fluid dynamics, and fusion energy sciences. As today's high-performance computing capabilities advance, these datasets often reach trillions of points, posing significant visualization, analysis, and storage challenges. While error-bounded lossy compression makes it possible to represent floating-point values with strict pointwise accuracy guarantees, the lack of…

https://slow-bros.com/harold-halibut
Harold Halibut (Multi, XP'd on PC) Experience life on FEDORA, a spaceship crashed and floating in an endless ocean, from the POV of Harold, Fedora's Everyman janitor.
Do you like Point & Click/Adventure games? Play this. It's that…

This https://arxiv.org/abs/2402.00466 has been replaced.
link: https://scholar.google.com/scholar?q=a

Towards a GPU-Parallelization of the neXtSIM-DG Dynamical Core
The cryosphere plays a significant role in Earth's climate system. Therefore, an accurate simulation of sea ice is of great importance to improve climate projections. To enable higher resolution simulations, graphics processing units (GPUs) have become increasingly attractive as they offer higher floating point peak performance and better energy efficiency compared to CPUs. However, making use of this theoretical peak performance, which is based on massive data parallelism, usually requires mor…

This https://arxiv.org/abs/2403.05465 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csAR_…

Algorithm-Hardware Co-Design of Distribution-Aware Logarithmic-Posit Encodings for Efficient DNN Inference
Traditional Deep Neural Network (DNN) quantization methods using integer, fixed-point, or floating-point data types struggle to capture diverse DNN parameter distributions at low precision, and often require large silicon overhead and intensive quantization-aware training. In this study, we introduce Logarithmic Posits (LP), an adaptive, hardware-friendly data type inspired by posits that dynamically adapts to DNN weight/activation distributions by parameterizing LP bit fields. We also develop …

Towards Fixed-Point Formats Determination for Faust Programs
Agathe HerrouGRAME, Florent de DinechinINSA Lyon, St\'ephane LetzGRAME, Yann Orlarey, Anastasia Volkova
https://arxiv.org/abs/2403.06527

Towards Fixed-Point Formats Determination for Faust Programs
Modern programmable digital signal processing relies on floating-point numbers for their ease of use. Fixed-point number formats have the potential to save resources and improve execution time, but realising this potential burdens the programmer with the need to define each format, at every step of the computation. This article reviews existing methods to automatically determine fixed-point formats, then describes and evaluates the prototype implementation of automatic fixed-point format determ…

Certified Constraint Propagation and Dual Proof Analysis in a Numerically Exact MIP Solver
Sander Borst, Leon Eifler, Ambros Gleixner
https://arxiv.org/abs/2403.13567

Certified Constraint Propagation and Dual Proof Analysis in a Numerically Exact MIP Solver
This paper presents the integration of constraint propagation and dual proof analysis in an exact, roundoff-error-free MIP solver. The authors employ safe rounding methods to ensure that all results remain provably correct, while sacrificing as little computational performance as possible in comparison to a pure floating-point implementation. The study also addresses the adaptation of certification techniques for correctness verification. Computational studies demonstrate the effectiveness of t…

A Precision-Optimized Fixed-Point Near-Memory Digital Processing Unit for Analog In-Memory Computing
Elena Ferro, Athanasios Vasilopoulos, Corey Lammie, Manuel Le Gallo, Luca Benini, Irem Boybat, Abu Sebastian
https://arxiv.org/abs/2402.07549

A Precision-Optimized Fixed-Point Near-Memory Digital Processing Unit for Analog In-Memory Computing
Analog In-Memory Computing (AIMC) is an emerging technology for fast and energy-efficient Deep Learning (DL) inference. However, a certain amount of digital post-processing is required to deal with circuit mismatches and non-idealities associated with the memory devices. Efficient near-memory digital logic is critical to retain the high area/energy efficiency and low latency of AIMC. Existing systems adopt Floating Point 16 (FP16) arithmetic with limited parallelization capability and high late…

This https://arxiv.org/abs/2401.17957 has been replaced.
link: https://scholar.google.com/scholar?q=a

Avoiding breakdown in incomplete factorizations in low precision arithmetic
The emergence of low precision floating-point arithmetic in computer hardware has led to a resurgence of interest in the use of mixed precision numerical linear algebra. For linear systems of equations, there has been renewed enthusiasm for mixed precision variants of iterative refinement. We consider the iterative solution of large sparse systems using incomplete factorization preconditioners. The focus is on the robust computation of such preconditioners in half precision arithmetic and emplo…

GPU-Based Parallel Computing Methods for Medical Photoacoustic Image Reconstruction
Xinyao Yi, Yuxin Qiao
https://arxiv.org/abs/2404.10928 https://<…

GPU-Based Parallel Computing Methods for Medical Photoacoustic Image Reconstruction
Recent years have witnessed a rapid advancement in GPU technology, establishing it as a formidable high-performance parallel computing technology with superior floating-point computational capabilities compared to traditional CPUs. This paper explores the application of this technology in the field of photoacoustic imaging, an emerging non-destructive testing technique in biomedical engineering characterized by its high contrast, resolution, and penetration depth. We conduct a data parallelism …

Switchable Single/Dual Edge Registers for Pipeline Architecture
Suyash Vardhan Singh, Rakeshkumar Mahto
https://arxiv.org/abs/2404.12306 https://

Switchable Single/Dual Edge Registers for Pipeline Architecture
The demand for low power processing is increasing due to mobile and portable devices. In a processor unit, an adder is an important building block since it is used in Floating Point Units (FPU) and Arithmetic Logic Units (ALU). Also, pipeline techniques are used extensively to improve the throughput of the processing unit. To implement a pipeline requires adding a register at each sub-stage that result in increasing the latency. Moreover, designing a low power pipeline adder with low latency ha…

Algorithm-Hardware Co-Design of Distribution-Aware Logarithmic-Posit Encodings for Efficient DNN Inference
Akshat Ramachandran, Zishen Wan, Geonhwa Jeong, John Gustafson, Tushar Krishna
https://arxiv.org/abs/2403.05465

Algorithm-Hardware Co-Design of Distribution-Aware Logarithmic-Posit Encodings for Efficient DNN Inference
Traditional Deep Neural Network (DNN) quantization methods using integer, fixed-point, or floating-point data types struggle to capture diverse DNN parameter distributions at low precision, and often require large silicon overhead and intensive quantization-aware training. In this study, we introduce Logarithmic Posits (LP), an adaptive, hardware-friendly data type inspired by posits that dynamically adapts to DNN weight/activation distributions by parameterizing LP bit fields. We also develop …

This https://arxiv.org/abs/2311.04362 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Fast and forward stable randomized algorithms for linear least-squares problems
Iterative sketching and sketch-and-precondition are randomized algorithms used for solving overdetermined linear least-squares problems. When implemented in exact arithmetic, these algorithms produce high-accuracy solutions to least-squares problems faster than standard direct methods based on QR factorization. Recently, Meier, Nakatsukasa, Townsend, and Webb demonstrated numerical instabilities in a version of sketch-and-precondition in floating point arithmetic (arXiv:2302.07202). The work of…

Algorithm-Hardware Co-Design of Distribution-Aware Logarithmic-Posit Encodings for Efficient DNN Inference
Akshat Ramachandran, Zishen Wan, Geonhwa Jeong, John Gustafson, Tushar Krishna
https://arxiv.org/abs/2403.05465

Algorithm-Hardware Co-Design of Distribution-Aware Logarithmic-Posit Encodings for Efficient DNN Inference
Traditional Deep Neural Network (DNN) quantization methods using integer, fixed-point, or floating-point data types struggle to capture diverse DNN parameter distributions at low precision, and often require large silicon overhead and intensive quantization-aware training. In this study, we introduce Logarithmic Posits (LP), an adaptive, hardware-friendly data type inspired by posits that dynamically adapts to DNN weight/activation distributions by parameterizing LP bit fields. We also develop …

This https://arxiv.org/abs/2305.06946 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csAR_…

Big-PERCIVAL: Exploring the Native Use of 64-Bit Posit Arithmetic in Scientific Computing
The accuracy requirements in many scientific computing workloads result in the use of double-precision floating-point arithmetic in the execution kernels. Nevertheless, emerging real-number representations, such as posit arithmetic, show promise in delivering even higher accuracy in such computations. In this work, we explore the native use of 64-bit posits in a series of numerical benchmarks and compare their timing performance, accuracy and hardware cost to IEEE 754 doubles. In addition, we a…